feat(miner-selfimprove): wire the historical-replay scorer into the Phase 7 calibration loop (#4248) by andriypolanski · Pull Request #5462 · JSONbored/loopover

andriypolanski · 2026-07-12T20:41:04Z

Summary

Adds the missing miner-side runner (runHistoricalReplayCalibrationCycle, packages/gittensory-miner/lib/calibration-run.js) that finally connects the two finished-but-unwired halves feat(miner): wire the historical-replay harness into the live Phase 7 calibration loop #3014 left apart: the deterministic replay scorer (computeObjectiveAnchor, feat(miner): objective-anchor scoring for replayed plans against revealed history #3012) and the engine's pure calibration combine contract (computePhase7CalibrationLoop, feat(miner): wire the historical-replay harness into the live Phase 7 calibration loop #3014/PR feat(engine): wire historical replay into Phase 7 calibration loop #3225).
The runner scores a completed replay run with the objective-anchor scorer, reduces the per-task scores to one composite [0, 1], folds it into the HistoricalReplayCalibrationInput shape the engine expects, calls computePhase7CalibrationLoop with that plus the existing pr_outcome signal, and persists the combined snapshot as a calibration_snapshot event on the local append-only event ledger (the same typed-event-over-event-ledger.js pattern as pr-outcome.js, feat(miner-manage): local pr_outcome record writer #4274).
The persisted metric is queryable with the existing tooling — gittensory-miner ledger list --type calibration_snapshot — and via the new readCalibrationSnapshots / latestCalibrationSnapshot readers. No new CLI surface was required.
Adds an integration test exercising the full chain (replay scorer → combining contract → persisted result) against the real engine combine and a real temp-file ledger, plus a fail-closed degraded-harness case.
Adds a "The runner" note to docs/miner-selfimprove-calibration.md clarifying that feat(miner): wire the historical-replay harness into the live Phase 7 calibration loop #3014 shipped only the engine combine and that this runner (feat(miner-selfimprove): wire the historical-replay scorers into the Phase 7 calibration loop #4248) is what actually wires the replay scorer into it — the requested scope-clarification so the gap does not recur.
Consistent with the phase's maintainer-only boundary, the runner is read/measure-only: it produces and persists the tracked metric but never acts on it (no autonomy-level bump, no gate-threshold tune — that enforcement stays maintainer-only, maintainer: wire calibration accuracy into the live auto-tune circuit-breaker for miner-originated PRs #2352).

Closes #4248.

Scope

The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

If any required check was skipped, explain why:

Run the full gate with npm run test:ci (and npm run test:coverage unsharded) under Node 22 (.nvmrc) before pushing. The development sandbox runs Node 18 without the workspace node_modules (the engine's dist/ is unbuilt and node:sqlite is unavailable), so the Vitest gate could not be executed here. The new module's pure logic was verified out-of-band with a standalone Node script against the real replay-objective-anchor.js scorer (composite of a fully-overlapping feature task = 1.0; combined with a 0.8 pr_outcome at default 0.5/0.5 weights = 0.9) and against verbatim copies of the snapshot/normalize helpers (every accept/reject/fallback branch).
Codecov surface: only packages/gittensory-miner/lib/calibration-run.js is measured (packages/gittensory-miner/lib/**); the .d.ts, package.json, docs, and test/** are not. calibration-run.js is written to be pure and dependency-injected, and test/unit/miner-calibration-run.test.ts targets 100% of its lines and branches (both sides of every guard, ??, and default parameter), with the integration test covering the default engine-combine + real-ledger paths.

Safety

No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed. The persisted snapshot surfaces only accuracies, the documented baseline, hold-reason codes, and provenance — never raw replay scores or rewards.
Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (None in this PR.)
API/OpenAPI/MCP behavior is updated and tested where needed. (No API/OpenAPI/MCP surface changed; the snapshot is queryable via the existing ledger list --type path.)
UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (No UI changes.)
Visible UI changes include a UI Evidence section below. (Not applicable — no UI changes.)
Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (CHANGELOG.md untouched.)

UI Evidence

Not applicable — this change is a miner-package runtime module + tests + package docs (Markdown). No rendered frontend surface changed.

Notes

Design boundary respected. computePhase7CalibrationLoop is a pure engine contract that cannot depend on the miner (miner → engine, not the reverse), so it correctly waited for an external caller. This runner is that caller and lives on the miner side; it measures and records only. Acting on the combined accuracy — the calibration-gated circuit-breaker — remains maintainer-only (maintainer: wire calibration accuracy into the live auto-tune circuit-breaker for miner-originated PRs #2352), as documented in docs/miner-selfimprove-calibration.md.
Fail-closed preserved. A degraded/unavailable replay harness flows through verbatim to the engine, whose fail-closed hold path blocks any autonomy increase even when pr_outcome has signal; the integration test pins this.
Persistence pattern. The calibration_snapshot typed event mirrors pr-outcome.js's MINER_PR_OUTCOME_EVENT over the shared append-only event-ledger.js, so it inherits the ledger's immutability/JSON-verbatim guarantees and is immediately queryable with gittensory-miner ledger list --type calibration_snapshot.

…hase 7 calibration loop (JSONbored#4248)

superagent-security · 2026-07-12T20:42:01Z

Superagent didn't find any vulnerabilities or security issues in this PR.

loopover-orb · 2026-07-12T20:42:36Z

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

_{Review updated: 2026-07-12 20:49:03 UTC}

7 files · 2 AI reviewers · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

safe to merge

Review summary
This PR adds the missing miner-side runner connecting the deterministic replay scorer to the engine's Phase7 calibration combine, persisting results as a typed event-ledger row exactly mirroring the existing pr-outcome.js pattern. The wiring is traced end-to-end in the integration test against the real engine combine and a real temp-file SQLite ledger, and the fail-closed degraded-harness case is exercised correctly (autonomyIncreasePermitted false, snapshot still persisted). The module is read/measure-only as claimed — no autonomy or threshold mutation paths are present — and the composite score correctly returns null rather than a fabricated 0 when there are no scorable tasks.

Nits — 5 non-blocking

packages/gittensory-miner/lib/calibration-run.js:120 — normalizeCalibrationSnapshotPayload has ~14 branches (threshold 10); consider extracting the per-field validators into a small table/loop for readability.
packages/gittensory-miner/lib/calibration-run.js:53 — scoreHistoricalReplayComposite nests to depth 5; an early-continue restructure would flatten the loop body.
packages/gittensory-miner/lib/calibration-run.js — the runner silently swallows a malformed snapshot when a ledger IS injected (recorded stays null with no signal to the caller); consider at least logging or surfacing that case since it means a calibration cycle silently fails to persist.
Extract the repeated isFiniteNumber/optionalString-guard pattern in normalizeCalibrationSnapshotPayload into a small declarative field-spec to shrink the complexity flagged above.
Consider whether recordCalibrationSnapshot returning null on a malformed payload should also emit a warning, since currently a caller has no way to distinguish 'not persisted because malformed' from any other silent no-op.

Signal	Result	Evidence
Code review	✅ No blockers	2 reviewers, synthesized
Linked issue	✅ Linked	#4248
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 155 registered-repo PR(s), 105 merged, 29 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	andriypolanski; Gittensor profile; 155 PR(s), 29 issue(s).
Gate result	✅ Passing	No configured blocker found.
Improvement	✅ Minor	risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — The diff directly closes the stated wiring gap by connecting existing replay scoring and engine combine logic to a queryable persisted snapshot without adding new command surface.

Review context

Author: andriypolanski
Role context: outside_contributor
Public audience mode: oss maintainer
Lane context: Repository is configured for direct PR review.
Public profile languages: not available
Official Gittensor activity: 155 PR(s), 29 issue(s).
PR-specific overlap: none found.

Contributor next steps

Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

Related work = same linked issue, overlapping active PRs, or title/path similarity.
Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

@gittensory ask <question> answers contribution-quality Q&A with source citations and freshness.
@gittensory chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

Re-run Gittensory review

codecov · 2026-07-12T20:44:17Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.71%. Comparing base (30a6ffb) to head (bdf618c).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5462      +/-   ##
==========================================
+ Coverage   94.52%   94.71%   +0.18%     
==========================================
  Files         554      555       +1     
  Lines       44469    44526      +57     
  Branches    14666    14666              
==========================================
+ Hits        42034    42172     +138     
+ Misses       1754     1619     -135     
- Partials      681      735      +54

Flag	Coverage Δ
shard-1	`43.89% <0.00%> (-0.18%)`	⬇️
shard-2	`35.17% <100.00%> (+0.17%)`	⬆️
shard-3	`31.65% <0.00%> (-0.35%)`	⬇️
shard-4	`31.80% <92.98%> (+0.11%)`	⬆️
shard-5	`32.85% <0.00%> (+0.14%)`	⬆️
shard-6	`44.05% <0.00%> (+0.51%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
packages/gittensory-miner/lib/calibration-run.js	`100.00% <100.00%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

loopover-orb

Gittensory approves — the gate is satisfied and CI is green.

andriy-polanski added 2 commits July 12, 2026 20:37

feat(miner-selfimprove): wire the historical-replay scorer into the P…

6a2445d

…hase 7 calibration loop (JSONbored#4248)

Merge branch 'main' into feat/miner-phase7-calibration-runner-4248

bdf618c

andriypolanski requested a review from JSONbored as a code owner July 12, 2026 20:41

andriypolanski marked this pull request as draft July 12, 2026 20:41

loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 12, 2026

github-actions Bot deployed to preview/pr-5462 July 12, 2026 20:42 View deployment

andriypolanski marked this pull request as ready for review July 12, 2026 20:47

loopover-orb Bot assigned andriypolanski Jul 12, 2026

loopover-orb Bot approved these changes Jul 12, 2026

View reviewed changes

loopover-orb Bot merged commit 7e2e20d into JSONbored:main Jul 12, 2026
16 checks passed

github-actions Bot mentioned this pull request Jul 14, 2026

chore(release): cut miner v1.0.0 #5712

Merged

andriypolanski deleted the feat/miner-phase7-calibration-runner-4248 branch July 16, 2026 15:11

Navigation Menu

Uh oh!

feat(miner-selfimprove): wire the historical-replay scorer into the Phase 7 calibration loop (#4248) - #5462

feat(miner-selfimprove): wire the historical-replay scorer into the Phase 7 calibration loop (#4248)#5462
loopover-orb[bot] merged 2 commits into
JSONbored:mainfrom
andriypolanski:feat/miner-phase7-calibration-runner-4248

andriypolanski commented Jul 12, 2026 •

edited

Loading

Uh oh!

superagent-security Bot commented Jul 12, 2026

Uh oh!

loopover-orb Bot commented Jul 12, 2026 •

edited

Loading

✅ Gittensory review result - approve/merge recommended

Uh oh!

codecov Bot commented Jul 12, 2026 •

edited

Loading

Uh oh!

loopover-orb Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

andriypolanski commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Scope

Validation

Safety

UI Evidence

Notes

Uh oh!

superagent-security Bot commented Jul 12, 2026

Uh oh!

loopover-orb Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Gittensory review result - approve/merge recommended

Uh oh!

codecov Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

loopover-orb Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andriypolanski commented Jul 12, 2026 •

edited

Loading

loopover-orb Bot commented Jul 12, 2026 •

edited

Loading

codecov Bot commented Jul 12, 2026 •

edited

Loading